Translation Methodology in the Spoken Language Translator: An Evaluation

In this paper we describe how the translation methodology adopted for the Spoken Language Translator (SLT) addresses the characteristics of the speech translation task in a context where it is essential to achieve easy customization to new languages and new domains. We then discuss the issues that arise in any attempt to evaluate a speech translator, and present the results of such an evaluation carried out on SLT for several language pairs. 1 The nature of the speech translation task Speech translation is in many respects a particularly difficult version of the translation task. High quality output is essential: the speech produced must sound natural if it is to be easily comprehensible. The quality of the translation itself must also be high, in spite of the fact that, by the nature of the problem, no post-editing is possible. Things are equally difficult on the input side: pre-editing, too, is difficult or impossible, yet ill-formed input and recognition errors are both likely to be quite common. Thus robust analysis and translation are also required. Furthermore, any attempted solutions to these problems must be capable of operating at a speed close enough to real time that users are not faced with unacceptable delays. Together, these factors mean that speech translation is currently only practical for limited domains, typically involving a vocabulary of a few thousand words. Because of this, it is desirable that a speech translator should be easily portable to new domains. Portability to new languages, involving the acquisition of both monolingual and cross-linguistic information, should also be as straightforward as possible. These ends can be achieved by using general-purpose components for both speech and language processing and training them on domain-specific speech and text corpora. The training should be automated whenever possible, and where human intervention is required, the process should be deskilled to the level where, ideally, it can be carried out by people who are familiar with the domain but are not experts in the systems themselves. These points will be discussed in the context of the Spoken Language Translator (SLT) (Rayner, Alshawi eta/, 1993; Agn~s et al., 1994; Rayner and Carter, 1997), a customizable speech translator built as a pipelined sequence of generalpurpose components. These components are: a version of the Decipher (TM) speech recognizer (Murveit eta/, 1993) for the source language; a copy of the Core Language Engine (CLE) (Alshawi (ed), 1992) for the source language; another copy of the CLE for the target language; and a target language text-to-speech synthesizer. The current SLT system carries out multilingual speech translation in near real time in the ATIS domain (Hemphill et al., 1990) for several language pairs. Good demonstration versions exist for the four pairs English ~ Swedish, English French, Swedish ~ English and Swedish Danish. Preliminary versions exist for five more pairs: Swedish ~ French, French --~ English, English ~ Danish, French --d. Spanish and English --~ Spanish. We describe the methodology used to build the SLT system itself, particularly in the areas of customization (Section 2), robustness (Section 3), and multilinguality (Section 4). For further details on the topics of customization and multilin-